{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/pinecone-io/examples/blob/master/learn/generation/pinecone-assistant/assistants-ai-demo.ipynb) [![Open nbviewer](https://raw.githubusercontent.com/pinecone-io/examples/master/assets/nbviewer-shield.svg)](https://nbviewer.org/github/pinecone-io/examples/blob/master/learn/generation/pinecone-assistant/assistants-ai-demo.ipynb)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ARZMWrqANRYw"
      },
      "source": [
        "# Pinecone Assistant Demo"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "kdx4h_jLM_dt"
      },
      "outputs": [],
      "source": [
        "!pip install -qU \\\n",
        "    pinecone-client==\"4.1.1\" \\\n",
        "    pinecone-plugin-assistant==\"0.1.3\" \\\n",
        "    pinecone-notebooks==\"0.1.1\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Tosz3yreNQsc"
      },
      "source": [
        "First, we initialize our connection to Pinecone, we first get our API key:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "zgQEFmwgNPqW"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "\n",
        "if not os.environ.get(\"PINECONE_API_KEY\"):\n",
        "    from pinecone_notebooks.colab import Authenticate\n",
        "    Authenticate()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "URRH71GhQRgb"
      },
      "source": [
        "Now initialize our client:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "rQb5-uUGPaj2"
      },
      "outputs": [],
      "source": [
        "from pinecone import Pinecone\n",
        "\n",
        "pc = Pinecone()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Hpi_buK4QYYI"
      },
      "source": [
        "To begin, we create a new assistant — you can name your assistant whatever you like, in this example we will build an assistant to help us review AI papers so we'll call this one `ai-researcher`. We can also add some additional metadata to our assistants, like who created the assistant and it's version number."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "wWLXlchYpUOc"
      },
      "outputs": [],
      "source": [
        "pc.assistant.list_assistants()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8bauAl4iu579"
      },
      "outputs": [],
      "source": [
        "name = \"airesearcher\""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "XJ_agTPqQX6h",
        "outputId": "e18cc135-00a5-48a1-e196-025210e897c7"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "[{'created_at': '2024-06-17T13:29:10Z',\n",
              "  'metadata': {'author': 'James Briggs', 'version': '0.1'},\n",
              "  'name': 'airesearcher',\n",
              "  'status': 'Ready',\n",
              "  'updated_at': '2024-06-17T13:29:12Z'}]"
            ]
          },
          "execution_count": 8,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "assistant = pc.assistant.create_assistant(\n",
        "    assistant_name=name,\n",
        "    metadata={\n",
        "        \"author\": \"James Briggs\",\n",
        "        \"version\": \"0.1\"\n",
        "    }\n",
        ")\n",
        "\n",
        "pc.assistant.list_assistants()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GppZbP3AJCJ-"
      },
      "source": [
        "We can either list all assistants as above, or show a single assistant as below:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "TM2fMg8lZ2x-",
        "outputId": "fb4bd9c6-a909-472d-f2ce-a8420eb25836"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'created_at': '2024-06-17T13:29:10Z',\n",
              " 'metadata': {'author': 'James Briggs', 'version': '0.1'},\n",
              " 'name': 'airesearcher',\n",
              " 'status': 'Ready',\n",
              " 'updated_at': '2024-06-17T13:29:12Z'}"
            ]
          },
          "execution_count": 9,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "pc.assistant.describe_assistant(assistant_name=name)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "id_y-PN_5pNh"
      },
      "source": [
        "We're not able to speak to the assistant before providing it with docs to power it's knowledge augmentation. If we try we'll see a `PineconeApiException`:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 463
        },
        "id": "pVte2-30uhDJ",
        "outputId": "31b08046-2b0e-43a9-9879-e4d26ba5ac55"
      },
      "outputs": [
        {
          "ename": "PineconeApiException",
          "evalue": "(400)\nReason: Bad Request\nHTTP response headers: HTTPHeaderDict({'content-type': 'text/plain; charset=utf-8', 'Content-Length': '98', 'access-control-allow-origin': '*', 'vary': 'origin,access-control-request-method,access-control-request-headers', 'access-control-expose-headers': '*', 'date': 'Mon, 17 Jun 2024 13:30:28 GMT', 'x-envoy-upstream-service-time': '197', 'server': 'envoy', 'Via': '1.1 google', 'Alt-Svc': 'h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000'})\nHTTP response body: {\"error\":{\"code\":\"INVALID_ARGUMENT\",\"message\":\"Assistant doesn't contain any files\"},\"status\":400}\n",
          "output_type": "error",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mPineconeApiException\u001b[0m                      Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-10-34ab3a126a86>\u001b[0m in \u001b[0;36m<cell line: 8>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      6\u001b[0m )\n\u001b[1;32m      7\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 8\u001b[0;31m \u001b[0massistant\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mchat_completions\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmessages\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mmsg\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/pinecone_plugins/assistant/models/assistant_model.py\u001b[0m in \u001b[0;36mchat_completions\u001b[0;34m(self, messages, stream)\u001b[0m\n\u001b[1;32m    315\u001b[0m             \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_chat_completions_streaming\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmessages\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mmessages\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    316\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 317\u001b[0;31m             \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_chat_completions_single\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmessages\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mmessages\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    318\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    319\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m_chat_completions_single\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmessages\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mList\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mMessage\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m->\u001b[0m \u001b[0mChatResultModel\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/pinecone_plugins/assistant/models/assistant_model.py\u001b[0m in \u001b[0;36m_chat_completions_single\u001b[0;34m(self, messages)\u001b[0m\n\u001b[1;32m    320\u001b[0m         \u001b[0mmessages\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mChatContext\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mrole\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mctx\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrole\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcontent\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mctx\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcontent\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mctx\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mmessages\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    321\u001b[0m         \u001b[0mcontext\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mChatRequest\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmessages\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mmessages\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 322\u001b[0;31m         search_result = self.assistant_data_api.chat_completion_assistant(\n\u001b[0m\u001b[1;32m    323\u001b[0m             \u001b[0massistant_name\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mname\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    324\u001b[0m             \u001b[0minline_object1\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mcontext\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/pinecone_plugins/assistant/data/core/client/api_client.py\u001b[0m in \u001b[0;36m__call__\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m    770\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    771\u001b[0m         \"\"\"\n\u001b[0;32m--> 772\u001b[0;31m         \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcallable\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    773\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    774\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mcall_with_http_info\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/pinecone_plugins/assistant/data/core/client/api/manage_assistants_api.py\u001b[0m in \u001b[0;36m__chat_completion_assistant\u001b[0;34m(self, assistant_name, inline_object1, **kwargs)\u001b[0m\n\u001b[1;32m    110\u001b[0m             \u001b[0mkwargs\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'inline_object1'\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m\u001b[0;31m \u001b[0m\u001b[0;31m\\\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    111\u001b[0m                 \u001b[0minline_object1\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 112\u001b[0;31m             \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcall_with_http_info\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    113\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    114\u001b[0m         self.chat_completion_assistant = _Endpoint(\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/pinecone_plugins/assistant/data/core/client/api_client.py\u001b[0m in \u001b[0;36mcall_with_http_info\u001b[0;34m(self, **kwargs)\u001b[0m\n\u001b[1;32m    832\u001b[0m             \u001b[0mparams\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'header'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'Content-Type'\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mheader_list\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    833\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 834\u001b[0;31m         return self.api_client.call_api(\n\u001b[0m\u001b[1;32m    835\u001b[0m             \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msettings\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'endpoint_path'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msettings\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'http_method'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    836\u001b[0m             \u001b[0mparams\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'path'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/pinecone_plugins/assistant/data/core/client/api_client.py\u001b[0m in \u001b[0;36mcall_api\u001b[0;34m(self, resource_path, method, path_params, query_params, header_params, body, post_params, files, response_type, auth_settings, async_req, _return_http_data_only, collection_formats, _preload_content, _request_timeout, _host, _check_type)\u001b[0m\n\u001b[1;32m    407\u001b[0m         \"\"\"\n\u001b[1;32m    408\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0masync_req\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 409\u001b[0;31m             return self.__call_api(resource_path, method,\n\u001b[0m\u001b[1;32m    410\u001b[0m                                    \u001b[0mpath_params\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mquery_params\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mheader_params\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    411\u001b[0m                                    \u001b[0mbody\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mpost_params\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfiles\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/pinecone_plugins/assistant/data/core/client/api_client.py\u001b[0m in \u001b[0;36m__call_api\u001b[0;34m(self, resource_path, method, path_params, query_params, header_params, body, post_params, files, response_type, auth_settings, _return_http_data_only, collection_formats, _preload_content, _request_timeout, _host, _check_type)\u001b[0m\n\u001b[1;32m    201\u001b[0m         \u001b[0;32mexcept\u001b[0m \u001b[0mPineconeApiException\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    202\u001b[0m             \u001b[0me\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mbody\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mbody\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdecode\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'utf-8'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 203\u001b[0;31m             \u001b[0;32mraise\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    204\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    205\u001b[0m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlast_response\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mresponse_data\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/pinecone_plugins/assistant/data/core/client/api_client.py\u001b[0m in \u001b[0;36m__call_api\u001b[0;34m(self, resource_path, method, path_params, query_params, header_params, body, post_params, files, response_type, auth_settings, _return_http_data_only, collection_formats, _preload_content, _request_timeout, _host, _check_type)\u001b[0m\n\u001b[1;32m    194\u001b[0m         \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    195\u001b[0m             \u001b[0;31m# perform request and return response\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 196\u001b[0;31m             response_data = self.request(\n\u001b[0m\u001b[1;32m    197\u001b[0m                 \u001b[0mmethod\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0murl\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mquery_params\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mquery_params\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mheaders\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mheader_params\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    198\u001b[0m                 \u001b[0mpost_params\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mpost_params\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mbody\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mbody\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/pinecone_plugins/assistant/data/core/client/api_client.py\u001b[0m in \u001b[0;36mrequest\u001b[0;34m(self, method, url, query_params, headers, post_params, body, _preload_content, _request_timeout)\u001b[0m\n\u001b[1;32m    453\u001b[0m                                             body=body)\n\u001b[1;32m    454\u001b[0m         \u001b[0;32melif\u001b[0m \u001b[0mmethod\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;34m\"POST\"\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 455\u001b[0;31m             return self.rest_client.POST(url,\n\u001b[0m\u001b[1;32m    456\u001b[0m                                          \u001b[0mquery_params\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mquery_params\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    457\u001b[0m                                          \u001b[0mheaders\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mheaders\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/pinecone_plugins/assistant/data/core/client/rest.py\u001b[0m in \u001b[0;36mPOST\u001b[0;34m(self, url, headers, query_params, post_params, body, _preload_content, _request_timeout)\u001b[0m\n\u001b[1;32m    300\u001b[0m     def POST(self, url, headers=None, query_params=None, post_params=None,\n\u001b[1;32m    301\u001b[0m              body=None, _preload_content=True, _request_timeout=None):\n\u001b[0;32m--> 302\u001b[0;31m         return self.request(\"POST\", url,\n\u001b[0m\u001b[1;32m    303\u001b[0m                             \u001b[0mheaders\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mheaders\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    304\u001b[0m                             \u001b[0mquery_params\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mquery_params\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/pinecone_plugins/assistant/data/core/client/rest.py\u001b[0m in \u001b[0;36mrequest\u001b[0;34m(self, method, url, query_params, headers, body, post_params, _preload_content, _request_timeout)\u001b[0m\n\u001b[1;32m    259\u001b[0m                 \u001b[0;32mraise\u001b[0m \u001b[0mServiceException\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mhttp_resp\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mr\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    260\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 261\u001b[0;31m             \u001b[0;32mraise\u001b[0m \u001b[0mPineconeApiException\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mhttp_resp\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mr\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    262\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    263\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0mr\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mPineconeApiException\u001b[0m: (400)\nReason: Bad Request\nHTTP response headers: HTTPHeaderDict({'content-type': 'text/plain; charset=utf-8', 'Content-Length': '98', 'access-control-allow-origin': '*', 'vary': 'origin,access-control-request-method,access-control-request-headers', 'access-control-expose-headers': '*', 'date': 'Mon, 17 Jun 2024 13:30:28 GMT', 'x-envoy-upstream-service-time': '197', 'server': 'envoy', 'Via': '1.1 google', 'Alt-Svc': 'h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000'})\nHTTP response body: {\"error\":{\"code\":\"INVALID_ARGUMENT\",\"message\":\"Assistant doesn't contain any files\"},\"status\":400}\n"
          ]
        }
      ],
      "source": [
        "from pinecone_plugins.assistant.models.chat import Message\n",
        "\n",
        "msg = Message(\n",
        "    content=\"tell me about the Mixtral 8x7B model\",\n",
        "    role=\"user\"  # either \"user\" or \"assistant\"\n",
        ")\n",
        "\n",
        "assistant.chat_completions(messages=[msg])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sR7ZSkur6LL6"
      },
      "source": [
        "So, let's go ahead and grab some PDFs. We'll download these from a GitHub repo containing some of the top AI papers from May-June 2024."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "B4ZPZDQQ553X",
        "outputId": "f16bf26c-525a-4ae6-8c45-71ba18538d33"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Cloning into 'assistants-ai-papers'...\n",
            "remote: Enumerating objects: 63, done.\u001b[K\n",
            "remote: Counting objects: 100% (3/3), done.\u001b[K\n",
            "remote: Total 63 (delta 0), reused 3 (delta 0), pack-reused 60\u001b[K\n",
            "Receiving objects: 100% (63/63), 355.34 MiB | 24.01 MiB/s, done.\n",
            "Updating files: 100% (58/58), done.\n"
          ]
        }
      ],
      "source": [
        "!git clone https://github.com/jamescalam/assistants-ai-papers.git"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FSit0pNl6r-X"
      },
      "source": [
        "We can find the docs here:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-BY8ash36iG-",
        "outputId": "75d61f43-c606-4881-b4fc-6e9d113be93d"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "['assistants-ai-papers/pdf/CRAG -- Comprehensive RAG Benchmark.pdf',\n",
              " \"assistants-ai-papers/pdf/Show, Don't Tell- Aligning Language Models with Demonstrated Feedback.pdf\",\n",
              " 'assistants-ai-papers/pdf/Llama 2- Open Foundation and Fine-Tuned Chat Models.pdf',\n",
              " 'assistants-ai-papers/pdf/Octopus v4- Graph of language models.pdf',\n",
              " 'assistants-ai-papers/pdf/Matryoshka Multimodal Models.pdf',\n",
              " 'assistants-ai-papers/pdf/Buffer of Thoughts- Thought-Augmented Reasoning with Large Language Models.pdf',\n",
              " 'assistants-ai-papers/pdf/Seed-TTS- A Family of High-Quality Versatile Speech Generation Models.pdf',\n",
              " 'assistants-ai-papers/pdf/The Prompt Report- A Systematic Survey of Prompting Techniques.pdf',\n",
              " 'assistants-ai-papers/pdf/OpenELM- An Efficient Language Model Family with Open-source Training and Inference Framework.pdf',\n",
              " 'assistants-ai-papers/pdf/MoRA- High-Rank Updating for Parameter-Efficient Fine-Tuning.pdf',\n",
              " 'assistants-ai-papers/pdf/To Believe or Not to Believe Your LLM.pdf',\n",
              " 'assistants-ai-papers/pdf/Prometheus 2- An Open Source Language Model Specialized in Evaluating Other Language Models.pdf',\n",
              " 'assistants-ai-papers/pdf/Autoregressive Model Beats Diffusion- Llama for Scalable Image Generation.pdf',\n",
              " 'assistants-ai-papers/pdf/LoRA Land- 310 Fine-tuned LLMs that Rival GPT-4, A Technical Report.pdf',\n",
              " 'assistants-ai-papers/pdf/LLMs achieve adult human performance on higher-order theory of mind tasks.pdf',\n",
              " 'assistants-ai-papers/pdf/Phi-3 Technical Report- A Highly Capable Language Model Locally on Your Phone.pdf',\n",
              " 'assistants-ai-papers/pdf/Multi-Head Mixture-of-Experts.pdf',\n",
              " 'assistants-ai-papers/pdf/Perplexed by Perplexity- Perplexity-Based Data Pruning With Small Reference Models.pdf',\n",
              " 'assistants-ai-papers/pdf/ShareGPT4Video- Improving Video Understanding and Generation with Better Captions.pdf',\n",
              " 'assistants-ai-papers/pdf/Replacing Judges with Juries- Evaluating LLM Generations with a Panel of Diverse Models.pdf',\n",
              " 'assistants-ai-papers/pdf/KAN- Kolmogorov-Arnold Networks.pdf',\n",
              " 'assistants-ai-papers/pdf/Mamba Linear-Time Sequence Modeling with Select State Spaces.pdf',\n",
              " 'assistants-ai-papers/pdf/Mixture-of-Agents Enhances Large Language Model Capabilities.pdf',\n",
              " 'assistants-ai-papers/pdf/Meteor- Mamba-based Traversal of Rationale for Large Language and Vision Models.pdf',\n",
              " 'assistants-ai-papers/pdf/Layer Skip- Enabling Early Exit Inference and Self-Speculative Decoding.pdf',\n",
              " 'assistants-ai-papers/pdf/Grokked Transformers are Implicit Reasoners- A Mechanistic Journey to the Edge of Generalization.pdf',\n",
              " 'assistants-ai-papers/pdf/Block Transformer- Global-to-Local Language Modeling for Fast Inference.pdf',\n",
              " 'assistants-ai-papers/pdf/How Far Are We to GPT-4V_ Closing the Gap to Commercial Multimodal Models with Open-Source Suites.pdf',\n",
              " 'assistants-ai-papers/pdf/What matters when building vision-language models_.pdf',\n",
              " 'assistants-ai-papers/pdf/Transformers are SSMs- Generalized Models and Efficient Algorithms Through Structured State Space Duality.pdf',\n",
              " 'assistants-ai-papers/pdf/Mixtral of Experts.pdf',\n",
              " 'assistants-ai-papers/pdf/RLHF Workflow- From Reward Modeling to Online RLHF.pdf',\n",
              " 'assistants-ai-papers/pdf/Introducing Super RAGs in Mistral 8x7B-v1.pdf',\n",
              " 'assistants-ai-papers/pdf/Your Transformer is Secretly Linear.pdf',\n",
              " 'assistants-ai-papers/pdf/Jina CLIP- Your CLIP Model Is Also Your Text Retriever.pdf',\n",
              " 'assistants-ai-papers/pdf/LLaMA- Open and Efficient Foundation Language Models.pdf',\n",
              " 'assistants-ai-papers/pdf/MAP-Neo- Highly Capable and Transparent Bilingual Large Language Model Series.pdf',\n",
              " 'assistants-ai-papers/pdf/MMLU-Pro- A More Robust and Challenging Multi-Task Language Understanding Benchmark.pdf',\n",
              " 'assistants-ai-papers/pdf/FIFO-Diffusion- Generating Infinite Videos from Text without Training.pdf',\n",
              " 'assistants-ai-papers/pdf/WildBench- Benchmarking LLMs with Challenging Tasks from Real Users in the Wild.pdf',\n",
              " 'assistants-ai-papers/pdf/ConvLLaVA- Hierarchical Backbones as Visual Encoder for Large Multimodal Models.pdf',\n",
              " 'assistants-ai-papers/pdf/Mistral 7B.pdf',\n",
              " 'assistants-ai-papers/pdf/Benchmarking Llama2, Mistral, Gemma and GPT for Factuality, Toxicity, Bias and Propensity for Hallucinations.pdf',\n",
              " 'assistants-ai-papers/pdf/An Introduction to Vision-Language Modeling.pdf',\n",
              " 'assistants-ai-papers/pdf/Transformers Can Do Arithmetic with the Right Embeddings.pdf',\n",
              " 'assistants-ai-papers/pdf/Make Your LLM Fully Utilize the Context.pdf',\n",
              " 'assistants-ai-papers/pdf/OpenRLHF- An Easy-to-use, Scalable and High-performance RLHF Framework.pdf',\n",
              " 'assistants-ai-papers/pdf/An Image is Worth 32 Tokens for Reconstruction and Generation.pdf']"
            ]
          },
          "execution_count": 12,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "from pathlib import Path\n",
        "\n",
        "pdf_paths = [str(p) for p in Path(\"./assistants-ai-papers/pdf\").glob(\"*.pdf\")]\n",
        "pdf_paths"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HTR8CCsg7ljH"
      },
      "source": [
        "We can upload these to our assistant like so:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8zGXOZ4c6zV-"
      },
      "outputs": [],
      "source": [
        "files = []\n",
        "\n",
        "for pdf_path in pdf_paths:\n",
        "    file_info = assistant.upload_file(\n",
        "        file_path=pdf_path,\n",
        "        timeout=-1  # return immediately and don't wait\n",
        "        # if `None`, waits until processing complete\n",
        "        # if >=`0`, time out after this many seconds\n",
        "    )\n",
        "    files.append(file_info)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PM5BFHQG8YQf"
      },
      "source": [
        "We can see the file info returned to us here:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "kXb42KQY8OtM",
        "outputId": "55484f0a-9ba8-45f4-ffa5-494831e33e7e"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "[{'created_on': '2024-06-17T13:32:45.900967622Z',\n",
              "  'id': '6142b982-e382-423a-85a8-fe2faa1fc16a',\n",
              "  'metadata': None,\n",
              "  'name': 'CRAG -- Comprehensive RAG Benchmark.pdf',\n",
              "  'size': 906174.0,\n",
              "  'status': 'Processing',\n",
              "  'updated_on': '2024-06-17T13:32:45.900967185Z'},\n",
              " {'created_on': '2024-06-17T13:32:46.985778099Z',\n",
              "  'id': 'e55af373-67fd-4143-a23a-3b01c18475a9',\n",
              "  'metadata': None,\n",
              "  'name': \"Show, Don't Tell- Aligning Language Models with Demonstrated \"\n",
              "          'Feedback.pdf',\n",
              "  'size': 1030722.0,\n",
              "  'status': 'Processing',\n",
              "  'updated_on': '2024-06-17T13:32:46.985777694Z'},\n",
              " {'created_on': '2024-06-17T13:32:48.269369985Z',\n",
              "  'id': 'b7e9a9bc-7c77-4bfb-9460-20b803e2d320',\n",
              "  'metadata': None,\n",
              "  'name': 'Llama 2- Open Foundation and Fine-Tuned Chat Models.pdf',\n",
              "  'size': 13661300.0,\n",
              "  'status': 'Processing',\n",
              "  'updated_on': '2024-06-17T13:32:48.269369349Z'}]"
            ]
          },
          "execution_count": 14,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "files[:3]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KS4fG1V-8dZ7"
      },
      "source": [
        "Because we set `timeout=-1` the `status` for all of these docs will show as `\"Processing\"`. We can check back on the document status using the `assistant.describe_file` method:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "MB7Jb0HO8VTX",
        "outputId": "39af1652-4144-46fd-cad9-a08a7886b515"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'created_on': '2024-06-17T13:32:45Z',\n",
              " 'id': '6142b982-e382-423a-85a8-fe2faa1fc16a',\n",
              " 'metadata': None,\n",
              " 'name': 'CRAG -- Comprehensive RAG Benchmark.pdf',\n",
              " 'size': 906174.0,\n",
              " 'status': 'Available',\n",
              " 'updated_on': '2024-06-17T13:33:15Z'}"
            ]
          },
          "execution_count": 15,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "assistant.describe_file(file_id=files[0].id)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kEanLRTh9Ayu"
      },
      "source": [
        "We can see that this document is now `\"Available\"`. We can confirm status for all of our files like so:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "0JkwJPwp89L-",
        "outputId": "d94d1c18-bbb1-453a-c8ea-82c266afcc6c"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "48 of 48 files are complete\n"
          ]
        }
      ],
      "source": [
        "complete = 0\n",
        "for file_info in files:\n",
        "    out = assistant.describe_file(file_id=file_info.id)\n",
        "    if out.status == \"Available\":\n",
        "        complete += 1\n",
        "\n",
        "print(f\"{complete} of {len(files)} files are complete\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Z4aql96a9cSF"
      },
      "source": [
        "All of our files have completed processing, so we can jump ahead into a conversation with our assistant about these files. First, let's confirm everything is working using the earlier message about Mixtral 8x7B:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 390
        },
        "id": "4DJkUQ7-9svn",
        "outputId": "d6e846c7-3339-4522-f625-b56ccb0429ed"
      },
      "outputs": [
        {
          "data": {
            "text/markdown": [
              "The Mixtral 8x7B model is a Sparse Mixture of Experts (SMoE) language model introduced by a team of researchers including Albert Q. Jiang, Alexandre Sablayrolles, and others. It is designed to enhance performance and efficiency in natural language processing tasks. The model architecture is based on the Mistral 7B but incorporates a unique feature where each layer consists of 8 feedforward blocks, known as \"experts.\" For each token processed, a router network selects two experts to handle the token, combining their outputs. This approach allows the model to access a total of 47 billion parameters while only using 13 billion active parameters during inference, which significantly reduces computational costs and latency  [1, p.1, 4].\n",
              "\n",
              "Mixtral 8x7B has been pretrained with multilingual data using a context size of 32,000 tokens, enabling it to perform well across various benchmarks. It outperforms or matches the performance of larger models like Llama 2 70B and GPT-3.5 on most benchmarks, particularly excelling in mathematics, code generation, and multilingual tasks  [1, p.1-2, 4]. The model also includes a fine-tuned version called Mixtral 8x7B – Instruct, which is optimized for following instructions and surpasses other models like GPT-3.5 Turbo, Claude-2.1, and Gemini Pro on human evaluation benchmarks  [1, p.1-2, 6].\n",
              "\n",
              "In terms of architecture, Mixtral 8x7B features 32 layers, a dimension size of 4096, 32 attention heads, and a hidden dimension of 14,336. The model supports a fully dense context length of 32,000 tokens and uses a vocabulary size of 32,000  [1, p.2]. The sparse mixture-of-experts mechanism allows for efficient computation by only activating a subset of the total parameters for each token, which helps in maintaining high performance while controlling computational costs  [1, p.1-2, 4].\n",
              "\n",
              "Overall, Mixtral 8x7B is notable for its efficiency and high performance across a range of tasks, making it a significant advancement in the field of language models  [1, p.1-2, 4, 6].\n",
              "\n",
              "References:\n",
              "1. [Mixtral of Experts.pdf](https://storage.googleapis.com/knowledge-engine-prod/airesearcher%2F141e59f0-c8f0-4f7f-91d8-74be3ca108cc.pdf?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=ke-prod-1%40pc-knowledge-prod.iam.gserviceaccount.com%2F20240617%2Fauto%2Fstorage%2Fgoog4_request&X-Goog-Date=20240617T133503Z&X-Goog-Expires=3600&X-Goog-SignedHeaders=host&response-content-disposition=inline&response-content-type=application%2Fpdf&X-Goog-Signature=b16aa2d352cf7156b0f441867b28b7d4f3b84bb73ab23d299015032d62a4f97de8c699335477d54bdd67edf1cbcd8600886100e7e7ffe8d8f7fe4d461c58f03accb32b184ebe6744075ad7b4c4ffb394044b6c53be80b6e5bc9d9bdf58d08f403ac46a236ef618d0c2313f40fc7d6d5702e5803b6e7b473b13e3a186be499b3418bf16448a4b0e7d142335dba9bae8c858a8d006ae90cc92db8a0dbdb49d1763b06f3c6e0633e8c6de204f948d476b7824ab766e77c172e29e364d51bc72ef725bd07b71319d83c01c443d629ec59fc4434ac4448286e38899eab8a101993c88713ff51a68a9c9c810be253d628a5b4ac95d95507d93eb57e2431e05cfd2d305)\n"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "execution_count": 17,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "from IPython.display import Markdown as md\n",
        "\n",
        "# quick check that we're not getting ahead of ourselves\n",
        "assert complete == len(files), \"make sure the above says '48 of 48 files are complete'\"\n",
        "\n",
        "# now run completion\n",
        "out = assistant.chat_completions(messages=[msg])\n",
        "md(out[\"choices\"][0][\"message\"][\"content\"])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oUncK9-aCkHg"
      },
      "source": [
        "Let's handle this as a conversation, ie storing interactions and sending our up-to-date chat history with every new interactions."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "V5vnqegpfo1Y"
      },
      "outputs": [],
      "source": [
        "https://storage.googleapis.com/knowledge-engine-prod/airesearcher%2F141e59f0-c8f0-4f7f-91d8-74be3ca108cc.pdf?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=ke-prod-1%40pc-knowledge-prod.iam.gserviceaccount.com%2F20240617%2Fauto%2Fstorage%2Fgoog4_request&X-Goog-Date=20240617T133503Z&X-Goog-Expires=3600&X-Goog-SignedHeaders=host&response-content-disposition=inline&response-content-type=application%2Fpdf&X-Goog-Signature=b16aa2d352cf7156b0f441867b28b7d4f3b84bb73ab23d299015032d62a4f97de8c699335477d54bdd67edf1cbcd8600886100e7e7ffe8d8f7fe4d461c58f03accb32b184ebe6744075ad7b4c4ffb394044b6c53be80b6e5bc9d9bdf58d08f403ac46a236ef618d0c2313f40fc7d6d5702e5803b6e7b473b13e3a186be499b3418bf16448a4b0e7d142335dba9bae8c858a8d006ae90cc92db8a0dbdb49d1763b06f3c6e0633e8c6de204f948d476b7824ab766e77c172e29e364d51bc72ef725bd07b71319d83c01c443d629ec59fc4434ac4448286e38899eab8a101993c88713ff51a68a9c9c810be253d628a5b4ac95d95507d93eb57e2431e05cfd2d305"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "cSp8p1evBp8m"
      },
      "outputs": [],
      "source": [
        "chat_history = [\n",
        "    msg,\n",
        "    Message(**out.choices[0].message.to_dict())\n",
        "]\n",
        "\n",
        "def chat(message: str):\n",
        "    # create Message object\n",
        "    msg = Message(content=message, role=\"user\")\n",
        "    # get response from assistant\n",
        "    out = assistant.chat_completions(messages=[msg])\n",
        "    assistant_msg = out.choices[0].message.to_dict()\n",
        "    # add to chat_history\n",
        "    chat_history.extend([msg, Message(**assistant_msg)])\n",
        "    return md(assistant_msg[\"content\"])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 222
        },
        "id": "7nl0TCIQEuEL",
        "outputId": "49801336-be3c-4012-be22-b7b73e88ef01"
      },
      "outputs": [
        {
          "data": {
            "text/markdown": [
              "Sparse Mixture of Experts (SMoE) is a model architecture in machine learning that enhances model capacity while maintaining a constant computational demand. This is achieved by activating only a small subset of experts (independent feed-forward networks) for each input, rather than using all experts simultaneously. This selective activation allows SMoE to perform better than densely-activated models on various tasks without significantly increasing training and inference costs  [1, p.1, 3].\n",
              "\n",
              "However, SMoE has some drawbacks, such as low expert activation, where only a small subset of experts are activated during optimization and inference, leading to suboptimal performance. Additionally, SMoE lacks fine-grained analytical capabilities for multiple semantic concepts within individual tokens  [1, p.1-2].\n",
              "\n",
              "References:\n",
              "1. [Multi-Head Mixture-of-Experts.pdf](https://storage.googleapis.com/knowledge-engine-prod/airesearcher%2F3908b358-601b-4e4a-900c-f4fd11f54afa.pdf?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=ke-prod-1%40pc-knowledge-prod.iam.gserviceaccount.com%2F20240617%2Fauto%2Fstorage%2Fgoog4_request&X-Goog-Date=20240617T133908Z&X-Goog-Expires=3600&X-Goog-SignedHeaders=host&response-content-disposition=inline&response-content-type=application%2Fpdf&X-Goog-Signature=b626ec962593dd020120a078bef82616f27c17ef737eba864cb3950b2be39fa595f885669392a6e57ba123015fd8cce91508e97f085d006e16ba0e5839da5600277339eccd0fc15d9b61289994a93ca3a44b96d291b7995770cf26b7a7942bb0b86b81c297f060253ed7ca63dbd5f7ec777a4b04569645e2ac27bf025930d3718e486a4e49be7f20461dc0f2f97bf36a9b7543f5c8b47cb5654af0c47ed27c7eda61d8574caac5608b81b0421075bf5f32be3b60dda09063d7e1fc97053e933fd039846a65cb900a72e0193d27e1a2b3887f5f5c3761d9b15973928c88e030a7623641ca5e016963fc215b25526b23b340406685c505b43cec476ef9a9a2193b)\n"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "execution_count": 21,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "chat(\"what does it mean by smoe?\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 384
        },
        "id": "eiLMKBkmFw5J",
        "outputId": "f88ebd5e-389a-4053-c7eb-2b13f4902f6a"
      },
      "outputs": [
        {
          "data": {
            "text/markdown": [
              "Low expert activation in Sparse Mixture of Experts (SMoE) models is considered detrimental for several reasons:\n",
              "\n",
              "1. **Underutilization of Model Capacity**: When only a small subset of experts is activated during optimization and inference, the model fails to utilize the full expressive power of all its experts. This underutilization is particularly problematic when the number of experts is large, as it significantly limits the model's effectiveness and scalability  [1, p.1-2].\n",
              "\n",
              "2. **Suboptimal Performance**: Low expert activation leads to suboptimal performance because the model does not leverage the diverse capabilities of all its experts. This limitation hinders the model's ability to learn and perform well on complex tasks that require a broader range of expertise  [1, p.1].\n",
              "\n",
              "3. **Inefficiency in Learning**: The low activation ratio means that many experts remain \"dead\" or inactive, which is inefficient. This inefficiency can be visualized in models like SMoE, where a significant portion of experts are not used at all, leading to wasted computational resources and potential  [1, p.1].\n",
              "\n",
              "4. **Limited Fine-Grained Understanding**: Low expert activation restricts the model's ability to capture multiple semantic concepts within individual tokens. This limitation affects the model's capacity to understand and process nuanced information, which is crucial for tasks involving complex language and vision patterns  [1, p.1-2].\n",
              "\n",
              "Overall, addressing low expert activation is essential for enhancing the model's performance, scalability, and ability to understand and process complex information effectively.\n",
              "\n",
              "References:\n",
              "1. [Multi-Head Mixture-of-Experts.pdf](https://storage.googleapis.com/knowledge-engine-prod/airesearcher%2F3908b358-601b-4e4a-900c-f4fd11f54afa.pdf?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=ke-prod-1%40pc-knowledge-prod.iam.gserviceaccount.com%2F20240617%2Fauto%2Fstorage%2Fgoog4_request&X-Goog-Date=20240617T134036Z&X-Goog-Expires=3600&X-Goog-SignedHeaders=host&response-content-disposition=inline&response-content-type=application%2Fpdf&X-Goog-Signature=877b4e526f80c550e8cc762aa736aec99ac004f3abd835e9a2e98cb6d71fb7710739b8b347b2f39cb90e0655dee11a0a4598c9375568708933176b79a38dab93b87291e8501df11df0d509dee8102d049272ed068350909154070b358e84dc0893f81592840cb747f953a41116c33a97be73f5e189f474abb3f598a9dcdf4e8ef3e410b9d1a9874d02befb1f0b88c7b2e6662691ea6b541b6a23c4edd59d439fd7c214cb1d26ebca4ac2352abd0732c056ae5573bfbcea138e5ee30e7b838a07e8dff59ff0c27b65681ec0ff44083c2c8bbd1322e427464572dea98dedc7ff613dc37fe9a683ccce55fe068f7bcb42dda6976c3adb2b487fed5b8ffb8c24e602)\n"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "execution_count": 22,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "chat(\"why is low expert activation a bad thing?\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 525
        },
        "id": "NmvHz2jNGJS7",
        "outputId": "f371c73e-7bf0-4492-d6e4-13da09d0852a"
      },
      "outputs": [
        {
          "data": {
            "text/markdown": [
              "Sure! Mamba-2 is a type of deep learning model designed to handle sequences of data, like text or audio, very efficiently. Here’s a simple breakdown:\n",
              "\n",
              "1. **What is Mamba-2?**\n",
              "   Mamba-2 is a sequence model that builds on the original Mamba model. It uses a special kind of layer called the Structured State Space (SSM) layer, which helps it process sequences more efficiently than traditional models like Transformers  [1, p.1, 3].\n",
              "\n",
              "2. **Why is it special?**\n",
              "   - **Efficiency**: Mamba-2 is designed to be faster and more efficient, especially for long sequences. It scales linearly with sequence length, meaning it can handle longer sequences without a huge increase in computational cost  [1, p.1, 3][2, p.2].\n",
              "   - **Performance**: It matches or even outperforms other models, including some that are twice its size, on various tasks like language modeling and associative recall tasks  [1, p.3, 28-30, 52].\n",
              "\n",
              "3. **How does it work?**\n",
              "   - **SSM Layers**: These layers help the model remember and process information over long sequences. Mamba-2 uses a refined version of these layers, called SSD (State Space Duality), which is faster and more efficient  [1, p.1-3].\n",
              "   - **Parallel Processing**: Mamba-2 uses parallel projections to handle data more efficiently, which helps in large-scale training and reduces the need for communication between different parts of the model  [1, p.3, 23, 26].\n",
              "\n",
              "4. **Hybrid Models**: Mamba-2 can be combined with other types of layers, like attention layers and MLP (Multi-Layer Perceptron) layers, to create hybrid models. These hybrids often perform better than using any single type of layer alone  [1, p.31].\n",
              "\n",
              "5. **Applications**: Mamba-2 is versatile and can be used for various tasks, including language modeling, audio processing, and even genomics. It has shown strong performance across these different domains  [2, p.1-2].\n",
              "\n",
              "In summary, Mamba-2 is a powerful and efficient model for handling sequences of data, leveraging advanced techniques to outperform many traditional models while being more computationally efficient.\n",
              "\n",
              "References:\n",
              "1. [Transformers are SSMs- Generalized Models and Efficient Algorithms Through Structured State Space Duality.pdf](https://storage.googleapis.com/knowledge-engine-prod/airesearcher%2F1927fb12-cece-43ac-8556-0c10d962876c.pdf?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=ke-prod-1%40pc-knowledge-prod.iam.gserviceaccount.com%2F20240617%2Fauto%2Fstorage%2Fgoog4_request&X-Goog-Date=20240617T134141Z&X-Goog-Expires=3600&X-Goog-SignedHeaders=host&response-content-disposition=inline&response-content-type=application%2Fpdf&X-Goog-Signature=037736f345803f4443b28ad4870a6caa5ec76d6498bce47ef36d710ae5b9b7a317afed4fe72b2398cfdcd292ce24835aaa2c399414b5a95d8f7adde3e8904f5d39eacfa9c520c5c813c8e1f80419f485f186de1811886ac1f8a32c4687fbb78af77fc6f01b272b4f25463be51f0f958c22918bede6b345a6c777277548ed349823c31b96733c86d8da557bc99d2848344a25296fb219111a7921c4ae839f68a2e0ac160328f6653c39f2b8ab6a2d392959c233a3490dbfe6b6d65798393c9153eca6ab1d3627127565add214a05637a8a7d7c70c1419ba5baf397c3ed4b55755c1019946c498f48282de2d5e6ec126326b1f012e0c5a20c53b1af46277884d4d)\n",
              "2. [Mamba Linear-Time Sequence Modeling with Select State Spaces.pdf](https://storage.googleapis.com/knowledge-engine-prod/airesearcher%2F453e8b8b-9ac7-4731-95b3-f67a7db7be3c.pdf?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=ke-prod-1%40pc-knowledge-prod.iam.gserviceaccount.com%2F20240617%2Fauto%2Fstorage%2Fgoog4_request&X-Goog-Date=20240617T134141Z&X-Goog-Expires=3600&X-Goog-SignedHeaders=host&response-content-disposition=inline&response-content-type=application%2Fpdf&X-Goog-Signature=7caaa0e49b72b3e49c99a5d23dc3b5f2bf077d6706ad1f8450d6b9400577f774d4d579594ad588afc0088b7f4a305cfdc8c68465a7677d34001b2b22bb2f2d09cef2cc9baf86382e07df4bb9df92833f431d0a566ffd35fbaedfec7ebac351a0d389ca674d1465a751e86742cef4e1b754743b4b1c7df4d106e904f7a51f4341ae277e47023ae87c8ff9f68659563eef9c424acaf20f6b7d58e3baed36e91d297e0b4a32564141ceee77905e3d085586045597f41a5acbc10c0a4b5d378a73fe57010e8ae3c26e993f4709e1b247bbd077deb4dba0242c2de1691d0a6d5472acf8278f8b37c1987d7467b3a57aca0bfe9fa7252ed6ac86a5dabc8c78d7cfb207)\n"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "execution_count": 23,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "chat(\n",
        "    \"interesting, thanks! I'd love to understand more about a different model \"\n",
        "    \"called Mamba-2, could you give me a ELI5 overview?\"\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "raVrzM_jLRYh"
      },
      "source": [
        "Once we're done with the assistant we can delete it like so:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "bHeVWUa_HR4i"
      },
      "outputs": [],
      "source": [
        "pc.assistant.delete_assistant(assistant_name=name)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HV7skmA0NKdG"
      },
      "source": [
        "---"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
